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We consider the effect of noise in sparse Boolean Networks with redundant functions. We show that they 
always exhibit a non-zero error level, and the dynamics undergoes a phase transition from non-ergodicity to 
ergodicity, as a function of noise, after which the system is no longer capable of preserving a memory of its 
initial state. We obtain upper-bounds on the critical value of noise for networks of different sparsity. 
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PACS numbers: 89.75.Da,05.65.+b,91.30.Dk,91.3().Px 

Introduction — Biological systems are unavoidably noisy in 
their nature, but often need to function in a predictable fash- 
ion HI. In such a situation, strategies to diminish the harm- 
ful effect of noise will significantly impact the fitness of a 
given organism. The most fundamental protection mechanism 
a system can adopt is the redundancy of its underlying com- 
ponents, since the resulting coincidences necessary to impact 
the proper function of the system can drastically diminish the 
probability of error. In this letter we are concerned with the ef- 
fect of redundancy in gene regulation; in particular in a simple 
Boolean Network (BN) model. We assume that each compo- 
nent in the system is arbitrarily redundant, with the only re- 
striction that the number of inputs per component is fixed and 
finite. In a general manner, we are able to show that redun- 
dancy can always guarantee reliable dynamics, up to a given 
critical value of noise, above which the system is incapable of 
maintaining any memory of its past states. From simple con- 
siderations, we are able to obtain upper bounds on the maxi- 
mum resilience attainable. This provides an important frame 
of reference to determine the reliability of a system with a 
given sparsity. 

We begin by defining the model, and how noise is intro- 
duced. A Boolean Network (BN) yj] is a directed graph with 
N nodes, representing the genes, which have an associated 
Boolean state U{ € {0, 1}, corresponding to the transcription 
state, and a function /i({oj};)> which determines the state of 
node i given the states of its input nodes {crj}i- The number 
of inputs of a given node is fcj, or simply k if its the same 
for all nodes. This system is usually updated in parallel, such 
that at each time step t, we have <Xj(i + 1) = fi({<Jj(t)}i). 
Starting from an initial configuration, the system will evolve, 
and eventually settle on an attractor. In a real system, the ex- 
pression level of a particular gene can fluctuate, despite the 
stability of its input states This characteristic can be in- 
corporated qualitatively in the BN model as uniform noise [0- 
rToh . defined as a probability p that, at each time step, the value 
of a given input <Tj € of a node i is flipped, prior to the 

evaluation of the function The value of p plays the role of 
a temperature in the system. If p = the original determin- 
istic model is recovered, and if p = 1/2 the system becomes 
effectively decoupled, with entirely stochastic dynamics. 

In the model above, it is known that error resilience does 
not spontaneously emerge, since Random Boolean Networks 
(BNs with random topology and functions iflllo . and simple 



functional elements such as loops always exhibit ergodic be- 
haviour in the presence of noise (p > 0) ifioll . To obtain re- 
silience, some level of functional redundancy must be intro- 
duced in the network. In the following we describe how this 
can be done, and analyse the optimal situation where all func- 
tions are arbitrarily redundant. From this situation we obtain 
upper bounds on the maximum reliability attainable, which 
is characterized by a transition from non-ergodic to ergodic 
behaviour at a critical noise value. 

Redundancy in sparse networks — Given a finite number of 
inputs per node k -C N, for any given (non-constant) func- 
tion, the probability that noise will change the output of the 
function will always be above zero, independent of the size 
of the network. Therefore, in average there will always be a 
non-vanishing fraction of the nodes which will be at the wrong 
state at any given time. The most that can be expected is that 
this fraction be as small as possible, and remain small as the 
dynamics evolve. The issue of achieving the first goal was 
first approached by von Neumann Il2lll3ll . who described a 
general mechanism of optimal redundancy, which is capable 
of reducing the propagation of errors in a BN. We will briefly 
outline this mechanism, and then show how it can be used to 
construct a dynamical model of error propagation in resilient 
Boolean Networks. 



The mechanism proposed in [ 12] consists of locally repli- 
cating a given function, such that the replicated input and out- 
put edges will form bundles which will all carry the same in- 
formation, in the absence of noise (see Fig [T}. The edges of 




FIG. 1: Redundancy construction method. Left: The original func- 
tion in the network; Middle: Equivalent redundant function, com- 
posed of the executive and restoring organs, with an edge bundle of 
size four. The grey rectangle corresponds to a random rewiring of 
the edges; Right: The resulting "pseudo-function", with incoming 
and outgoing edge bundles. 
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the output bundle are then randomized and fed into appro- 
priate restoration functions which will independently query 
the majority state carried in them. The number of edges in 
the bundle can be arbitrarily large, but the number of inputs 
per node must remain fixed. The output bundle of the last 
functions will then propagate the information to the rest of 
the network, which is also modified in the same manner. The 
first stage was dubbed in [ 12] the "executive organ", and the 
second stage, the "restoring organ". We note that while this 
method outlines a specific construction, it has a general na- 
ture, since it incorporates the two the most necessary features 
to be resilient against noise: replication and restoration of ma- 
jority values. It does so piecewise for all functions in a given 
network, depicting an alternative version with an optimal level 
of redundancy. This robust version will then function exactly 
like the original network, if each executive and restoring or- 
gan is thought of as an individual function (which we will 
call a pseudo-function). However, in the presence of noise, 
the fraction b of outputs having value 1 (and conversely 1 — 6 
with value 0) that enter and leave such a pseudo-function is 
no longer a Boolean value, but instead are real values in the 
range [0, 1], which will be continuously distributed in the limit 
of large number of edge in the bundles. In the following, we 
assume that the number of edges in the bundle is large enough, 
so that the fluctuations of the values of hi can be neglected. 
In this case, these pseudo-functions will be generalizations of 
the original Boolean function (plus restoration) in the real do- 
main, which will regulate how noise is propagated in the net- 
work. The general form of those pseudo-functions is 
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where Sij is the Kronecker delta, b{ is the input bundle i, f is 
the function of the executive organ, the variable j represents 
a given Boolean input combination, ji the i-th bit of the in- 
put combination j, and r(b) performs the restoring function 
(which we will describe in detail below). 

We are interested in analysing how the above mechanism 
can hinder error propagation on the network. The effect of 
noise can be measured in a variety of ways, but here we are 
interested in the ability of the system in remembering its past 
states, which we will label as non-ergodicity. More precisely, 
we can define an order parameter, the long-term hamming dis- 
tance, 
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where bi(t\a) is the value of b for pseudo-node i at time t, 
with a starting state bi(0\a) = a. The average (...) is taken 
over the whole network, and several independent realizations 
of the dynamics, with different initial states {erf} and {of}. If 
the system shows ergodic dynamics (as previously discussed), 
the value of h should converge to zero, corresponding to only 



one possible fixed point in the values of hi. Otherwise, it 
should decrease with the noise strength p, as the effects of 
noise brings the system closer to the ergodic phase 11711 . 

We will consider separately the case of networks with func- 
tions k = 2, and later the case k > 2. The case k = 1 will 
not be analysed since it does not allow for the construction of 
a restoring organ. 

k = 2 — The most crucial part in the procedure outlined 
above is the selection of the function to be used in the restor- 
ing organ. The function must be able to transform the values 
of the majority of the edges in the input bundle, into a even 
greater majority in the output bundle. However, no k = 2 
function can act as a simple majority function. There are how- 
ever some functions which behave as a majority function for 
certain input combinations, but not for others. These functions 
are the AND (8), NAND (7), OR (14) and NOR (1) which 
react only to two simultaneous input changes, if the original 
inputs are all 1 or 0, but react to any input flip if the origi- 
nal inputs are in the opposite state. Therefore those functions 
would be able to correct either value passing on the bundle, 
but not both. The solution proposed in ll 211 is to construct the 
restoring organ with two modules connected in sequence, both 
with either NAND or NOR functions 11811 . as can be seen in 
Fig. [2] The first tier will correct one of the values if it can 
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FIG. 2: Restoration organs with k = 2, with NAND (left) and NOR 
(right) functions. Below each are the restoration maps, for some val- 
ues of p. The dashed lines correspond to birf) = r\. 

and intrinsically flip the majority value of the bundle, and the 
second tier will then have its chance of correcting, now that 
the majority value has changed. We can verify the actual re- 
sponse of this scheme to noise, by defining two maps. First, 
the actual noise on the bundle, 



e (v) = (!- 2p)v+P, 



(3) 



where e is the fraction of the edges in the bundle with a given 
value, given the original fraction of same value r\. Second, the 
response of the NOR and NAND functions, 
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where b is the fraction of edges in the output bundle with value 
1, and Tj the fraction of inputs with the same value. The full 
restoration map is then given by 



b r (v) = (b 
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which is plotted in Fig. [2] for some values of p. We can see 
that the majority value on the bundle is preserved, even for 
non-zero values of p. However, the question remains if this 
restoration will be enough to maintain trajectories of a net- 
work from diverging. For that, we need to couple the restora- 
tion map above with the functions present on the network and 
iterate the system. However, there is one specific situation 
which represents the limiting case of maximum resilience, 
namely when after each pseudo-function there are infinitely 
many restoring organs in sequence. In this case, the response 
to noise of the function in the executive organ can be ne- 
glected, and the (infinitely long) restoring organ alone will 
determine the resilience of the network. Conveniently, this 
can be done with successive iterations Eq. [5] which should 
eventually reach a fixed point, corresponding to the roots of 
the equation (5 nol/nand e) 2 (i]) = r\. This is a fourth order 
polynomial in 77, and the roots can be obtained analytically. 
The system exhibits a typical pitchfork bifurcation, with three 
distinct fixed points (only two of each are stable), up to a crit- 
ical value of noise p c = (3 - \/7)/4 ps 0.0886, above which 
only one fixed point exists. This corresponds to a dynami- 
cal phase transition, where below this critical point the values 
of the outputs will oscillate between values close to and 1, 
and thus memory of the past states will always be preserved. 
Above the critical point, the dynamics is ergodic, independent 
of any starting state. In order to characterize this transition 
more precisely, we can write the expression for the previously 
defined order parameter in Eq.|2]as 



h= Iim[6(t|l)-6(t|0)] 
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where b(t\cr) is the value of b(t) with the starting point 6(0) = 
a, and p* = (3 + \/7)/4. The values of the order parameter 
are plotted in Fig [3] From Eq. [6] we also see easily that the 
critical exponent is 1/2 (mean-field universality class). The 




0.0 0.1 0.2 0.3 0.4 0.5 

V 



0.0 0.1 0.2 0.3 0.4 0.5 

V 



1.0 
* 0.5 
0.0 



0.0 



0.1 



0.2 



0.3 



0.4 



0.5 



FIG. 3: Fixed points, or period-2 points, b(oo) (top) and long-term 
hamming distance h (bottom) for the NAND (black) and NOR (gray) 
maps, as a function of noise. Black curves correspond to NAND 
restoration and grey ones to NOR. 

existence of this critical value of noise points to a direct up- 



per bound on the reliability attainable by k = 2 networks, 
since it represents the maximum limit of error correction of 
the restoring organ. Additionally, this critical value of noise 
corresponds exactly to the upped bound found rigorously by 
Evans and Pippinger | l3l for reliable computation of Boolean 
formulas composed of noisy NAND gates. 

k > 2 — If the functions have k > 2, the choice of the 
restoring organ becomes more obvious, and the most natural 
choice is the majority function, which returns simply the ma- 
jority value of its inputs. Since it will work equally well if 
the value on the bundle is either or 1, the majority func- 
tion is capable of performing restoration with only one tier of 
functions, without accumulating noise in an intermediate step, 
which provides it with superior characteristics. The restora- 
tion map of the majority function is given by 
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The last term is added only for functions with even k, which 
have an indeterminate majority state. In this case, it is as- 
sumed that half the restoring functions output 1 and the other 
half 0. It is also clear that majority functions with even k > 2 
will perform just as well as a k — 1 odd function, and there- 
fore the extra input is, for this purpose, wasted. We can anal- 
yse the quality of this restoration by iterating Eqs. [8] and [3] 
in sequence, like it was done for k = 2. In the absence of 
noise, this will lead to one of two fixed points, depending on 
the starting condition. It can be seen in Fig.|4]that those fixed 
points also merge into one at a critical value of noise, and the 
associate order parameter h also indicates a second order tran- 
sition, with the same critical exponent, but different critical 
noise values. As expected, the value of h is larger for larger k, 
for the same value p, and the critical noise is also larger. The 
critical values p c match exactly the upper bounds for Boolean 
formulas using noisy majority functions of k inputs found by 
Evans and Schulman [15], given by, 

1 2 k - 2 
Pc = 77 - T7X-TT, (9) 



k(k-i) ' 



for odd k. This scales with (1/2 — p c ) ~ 1/Vk (see Fig. [5]), 
leading to a strictly resilient situation when k — > 00. 

One can test the effectiveness of the majority functions, by 
considering other functions as the executive organ. Instead of 
systematically analysing all 2 2 functions, we can consider 
functions which are pathological in their stability to noise. 
Here we will consider a function with maximum sensitivity, 
which output or 1 if all inputs are 1 or 0, respectively, but 
otherwise the function is uniformly distributed, and the out- 
puts will be or 1 with equal probability for each input com- 
bination. The corresponding map can be written as 
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Its properties can be seen in Fig. [4] We see that indeed it 
becomes progressively difficult to stabilize for larger fc, and 
the critical point now scales as p c ~ 1/k, and the transition 
becomes first-order. On the other hand, the mere existence 
of the critical point confirms some level of resilience, which 
is not present in the system without redundancy, where the 
critical point is always p c = 0. 
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FIG. 4: Fixed points and long-term hamming distance, for the ma- 
jority restoring organ (left), and the maximum sensitivity executive 
organ with the majority restoring organ (right). 
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FIG. 5: Critical noise value p c as a function of fc, for the major- 
ity restoring organ, and k = 2 NAND/NOR restoration (left), and 
the maximum sensitivity executive organ with the majority restoring 
organ (right). The dashed lines correspond to \j\fk (left) and 1/k 
(right). 

Conclusion — We have shown that sparse networks, while 
they cannot be arbitrarily resilient, they can have stable dy- 
namics in the presence of noise, if redundancy is correctly 
introduced. The stability is marked by second or first-order 
transitions, from non-ergodic to ergodic behaviour. We obtain 
upper-bounds on the error resilience attainable by redundant 
networks with a given fc. This is in stark contrast to what is 
observed in Random Boolean Networks [10:], which never ex- 
hibit memory of its past states when noise is introduced, either 
in its frozen or "chaotic" phases. 

We have shown that the stabilization through redundancy is 
successful even with the most pathologically sensitive func- 
tions, such as the function with maximum sensitivity dis- 
cussed. We note also that redundancy provides additional ben- 
efits, such as robustness against damage and mutations, as was 



shown in 01611 . We stipulate that due to these robust features, 
redundancy must be present in some extent in real gene reg- 
ulatory networks; if not in the entire network, at least in its 
more dynamically relevant modules. On the other hand, arbi- 
trary redundancy close to the optimal bound is very unlikely 
due to its high putative cost to the organism, which would fa- 
vor instead a genetic circuit composed of fewer elements, with 
only enough resilience sufficient for survival. It remains to be 
seen to what extent is redundancy desirable, and how it may be 
connected with other topological and functional restrictions of 
gene regulation. 
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